Three-dimensional (3D) video coding is developed for encoding or decoding video data of multiple views simultaneously captured by multiple cameras. Since all cameras capture the same scene for both the texture data and depth data, there is a large amount of redundancy between texture and depth data in the same view. According to HEVC, the coding process is applied to each block, named as coding unit (CU) and each CU is subject to prediction process. Each CU may be divided into one or more prediction units (PUs) and the prediction process, such as Intra or Inter prediction, is applied to each PU. To reduce the redundancy, a texture-partition-dependent depth partition is utilized for processing a current depth block collocated with a collocated texture CU at the same quad-tree level according to the current 3D-HEVC (3D video coding based on high efficiency video coding (HEVC)).
In order to exploit the inter-view and inter-component redundancy, additional tools such as inter-view motion prediction (IVMP) and motion parameter inheritance (MPI) have been integrated to existing 3D-HEVC (High Efficiency Video Coding) and 3D-AVC (Advanced Video Coding) codec. To further improve the coding efficiency, sub-PU level IVMP and sub-PU level MPI are used. According to IVMP, the motion information associated with a block in a dependent view is predicted by the motion information associated with a corresponding block located using a disparity vector in a reference view. According to MPI, inheritance of the texture's motion parameters for depth data is achieved by adding one more Merge candidate to the Merge list of the current depth block, in addition to the usual spatial and temporal candidates from the HEVC Merge mode. The extra candidate is generated from the motion information of the co-located texture block.
In current 3D-HEVC, a syntax element iv_mv_pred_flag[layerId] is sent for each layer with layer ID larger than zero to indicate whether IVMP is used, where the layer ID larger than zero corresponds to a dependent view. For each layer with layer ID larger than zero, another syntax element log 2_sub_pb_size_minus3[layerId] is sent to indicate the texture sub-PU size. For depth coding, a syntax element mpi_flag[layerId] is sent for each layer with layer ID larger than zero to indicate whether MPI is used. However, the sub-PU size for MPI is shared by all layers and is indicated by the syntax element log 2_mpi_sub_pb_size_minus3 in video parameter set (VPS) extension. Table 1 shows the current vps_extension2( ) syntax design in 3D-HEVC.
TABLE 1De-scrip-vps_extension2( ) {torNote while( !byte_aligned( ) )  vps_extension_byte_alignment_reserved_one_bitu(1) for( i = 0; i <= vps_max_layers_minus1; i++ ) {  layerId = layer_id_in_nuh[ i ]  if ( layerId != 0) {   iv_mv_pred_flag[ layerId ]u(1)1-1   log2_sub_pb_size_minus3[ layerId ]ue(v)1-2   if ( !VpsDepthFlag[ layerId ] ) {    iv_res_pred_flag[ layerId ]u(1)    depth_refinement_flag[ layerId ]u(1)    view_synthesis_pred_flag[ layerId ]u(1)    depth_based_blk_part_flag[ layerId ]u(1)   } else {    mpi_flag[ layerId ]u(1)1-3    vps_depth_modes_flag[ layerId ]u(1)    lim_qt_pred_flag[ layerId ]u(1)    vps_inter_sdc_flag[ layerId ]u(1)   }  } } ... iv_mv_scaling_flagu(1) log2_mpi_sub_pb_size_minus3ue(v)1-4}
As shown in Table 1, the syntax elements related to iv_mv_pred_flag[layerId], log 2_sub_pb_size_minus3[layerId], mpi_flag[layerId] and log 2_mpi_sub_pb_size_minus3 are signaled in video parameter set extension, vps_extension2( ). The locations of these syntax elements are indicated by notes 1-1 through 1-4 respectively in Table 1. As is known that IVMP is used for texture data to take advantage of correlation between different views. On the other hand, MPI is used for depth data to take advantage of dependency of depth data on co-located texture data. It is noted that syntax elements log 2_sub_pb_size_minus3[layerId] and log 2_mpi_sub_pb_size_minus3 are signaled in the VPS independently without considering whether the current video data is texture data or depth data. Therefore, it is desirable to develop new method for sub-PU syntax element signaling to improve the performance.
In the current 3D-HEVC, illumination compensation (IC) is adopted to compensate the difference of illumination intensity between different views. When the IC is applied, the prediction value, y of a current block (110) in the current view is calculated based on a linear function (120) corresponding to y=a*x+b, where x is a sample value in the reference block in the reference view. In the linear model, parameter a is named as a multiplicative term and parameter b is named as an offset term. The two parameters a and b are derived according to training process (130) using the neighboring reconstructed samples (140) of the current block and the neighboring reconstructed samples (150) of the reference block (160) as depicted FIG. 1.
Samples involved in the training process according to HEVC are shown in FIG. 2, where a neighboring reference sample xi of the reference block (160) and a neighboring sample yi of the current block (110) possessing the same relative position (i.e., co-location) are used as a training pair. To reduce the number of training pairs, only one of every two adjacent samples are involved in the training set. The neighboring samples without labelling (i.e., xi or yi) are not used in the training process. The superscript “A” associated with xi and yi indicates that the neighboring samples are “Above” the block. The superscript “L” associated with xi and yi indicates that the neighboring samples are “Left” to the block.
The number of training pairs is proportional to the block size. For example, there are 8 training pairs for an 8×8 block and 64 training pairs for a 64×64 block. Therefore, the training process will be more complex for larger blocks.
The IC is applied to each color component, such as Y (Luma), U (Cb) and V (Cr) separately. The training process is also performed separately. The parameters a and b are trained independently for each component.
The basic procedures for calculating a and b are shown in the following equations:
                              a          =                                                                      ∑                                      i                    =                    0                                                        N                    -                    1                                                  ⁢                                                                  ⁢                                                      y                    ⁡                                          (                      i                      )                                                        ⁢                                      x                    ⁡                                          (                      i                      )                                                                                  -                                                ∑                                      i                    =                    0                                                        N                    -                    1                                                  ⁢                                                                  ⁢                                                      y                    ⁡                                          (                      i                      )                                                        ⁢                                                            ∑                                              i                        =                        0                                                                    N                        -                        1                                                              ⁢                                                                                  ⁢                                          x                      ⁡                                              (                        i                        )                                                                                                                                                                  ∑                                      i                    =                    0                                                        N                    -                    1                                                  ⁢                                                                  ⁢                                                      x                    ⁡                                          (                      i                      )                                                        ⁢                                      x                    ⁡                                          (                      i                      )                                                                                  -                                                ∑                                      i                    =                    0                                                        N                    -                    1                                                  ⁢                                                                  ⁢                                                      x                    ⁡                                          (                      i                      )                                                        ⁢                                                            ∑                      i                                              N                        -                        1                                                              ⁢                                                                                  ⁢                                          x                      ⁡                                              (                        i                        )                                                                                                                                ,                                  ⁢        and                            (        1        )                                b        =                                            ∑                              i                =                0                                            N                -                1                                      ⁢                                                  ⁢                          y              ⁡                              (                i                )                                              -                      a            ×                                          ∑                                  i                  =                  0                                                  N                  -                  1                                            ⁢                                                          ⁢                                                x                  ⁡                                      (                    i                    )                                                  .                                                                        (        2        )            
As depicted above, the complexity of IC is mainly due to the linear least square (LLS) method utilized for training a better a and b in both encoder and decoder. The LLS based training process requires multiple regular multiplication operations. According to the existing 3D-HEVC or 3D-AVC, the IC process is utilized for both uni-direction prediction and bi-directional prediction for both luminance and chroma components. Therefore, the IC process according to the existing 3D-HEVC or 3D-AVC requires intensive computations due to the multiplication operations. Therefore, it is desirable to simplify the IC process, particularly the LLS procedures involved.
In 3D video coding, a coding tool named texture-dependent depth partition (TDDP) has been developed for depth partition by taking advantage of correlation between the texture data and the depth data. To reduce the redundancy, the TDDP is utilized when the collocated coding unit (CU) of the current depth CU is at the same quad-tree level in the current 3D-HEVC.
FIG. 3 illustrates the correspondence between the allowed depth partitions and the corresponding texture partitions according to the TDDP of existing 3D-HEVC, where AMP (asymmetric motion partition is allowed). The texture partitions are shown on the left-hand side and the depth partitions are shown on the right-hand side. As shown in FIG. 3, when the collocated texture CU at the same quad-tree level is coded with vertical partitions, only vertical partitions and 2N×2N partition are allowed for the corresponding depth CU. Similarly, when the collocated texture CU at the same quad-tree level is coded with horizontal partitions, only horizontal partitions and 2N×2N partition are allowed for the corresponding depth CU. When the collocated texture CU at the same quad-tree level is coded with the N×N partition, the corresponding depth CU can use all partitions including both vertical and horizontal partitions, 2N×2N partition and N×N partition. When the collocated texture CU at the same quad-tree level is coded with the 2N×2N partition, the corresponding depth CU can only use the 2N×2N partition. As shown in FIG. 3, the vertical partitions include N×2N, nL×2N and nR×2N and the horizontal partitions include N×2N, nL×2N and nR×2N. Tables 2 and 3 show the two kinds of possible correspondences between part_mode values and prediction partition for Inter prediction. Table 2 presents the case that the partition mode only depends on part_mode. For Intra coding, 2N×2N and N×N partitions are allowed and part_mode has a value being 0 or 1. Table 3 shows the case that the partition mode depends on both part_mode and a partition variable partPredIdc. The partition variable partPredIdc can be interpreted as a texture partition type, where partPredIdc equal 1 represents horizontal partition, partPredIdc equal 2 represents vertical partition and partPredIdc equal 0 represents other partitions (i.e., 2N×2N or N×N).
TABLE 2part_modePartModeIntra coding0PART_2N×2Nmodes1PART_N×NInter coding0PART_2N×2Nmodes1PART_2N×N2PART_N×2N3PART_N×N4PART_2N×nU5PART_2N×nD6PART_nL×2N7PART_nR×2N
TABLE 3part_modepartPredIdcPartModeIntra coding00PART_2N×2Nmodes10PART_N×NInter coding00, 1, 2PART_2N×2Nmodes10PART_2N×N11PART_2N×N12PART_N×2N20PART_N×2N21PART_2N×nU22PART_nL×2N30PART_N×N31PART_2N×nD32PART_nR×2N40PART_2N×nU50PART_2N×nD60PART_nL×2N70PART_nR×2N
In semantics, when the collocated texture CU at the same quad-tree level is coded with horizontal or vertical, the variable partPredIdc is set equal to 1 or 2 respectively. When the collocated texture block has a larger depth or non-horizontal-and-vertical partitions, partPredIdc is set equal 0. In this way, the part_mode flag, which indicates the partition mode, can be either encoded with shorter bin string or the same as HEVC does.
However, in the existing 3D-HEVC (i.e., Working Draft 6), there is validity checking on the partition mode for the depth data. In particular, the existing 3D-HEVC only checks whether the current mode is an Intra or Inter mode and whether AMP (asymmetric motion partition) mode is enabled. Consequently, an illegal partition mode may be received at a decoder, while TDDP is utilized and partPredIdc is not equal to 0. Therefore, it is desirable to develop a partition mode coding for TDDP to ensure no illegal partition mode may occur or to ensure the decoder still operates properly in case of an illegal partition mode is received.